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ABSTRACT: In practice, it is desired to have estimates that are invariant under reparameterization. The 
' ' invariance property of the estimators helps to formulate a unified solution to the underlying estimation problem. 

In robust Bayesian analysis, a frequent criticism is that the optimal estimators are not invariant under smooth 
' reparameterizations. This paper considers the problem of posterior regret gamma-minimax (PRGM) estimation 
of the natural parameter of the exponential family of distributions under intrinsic loss functions. We show that 
, under the class of Jeffrey's Conjugate Prior (JCP) distributions, PRGM estimators are invariant to smooth 
, one-to-one reparameterizations. We apply our results to several distributions and different classes of JCP, as 
^NJ I well as the usual conjugate prior distributions. We observe that, in many cases, invariant PRGM estimators in 
the class of JCP distributions can be obtained by some modifications of PRGM estimators in the usual class of 
' conjugate priors. Moreover, when the class of priors are convex or dependant on a hyper-parameter belonging to 
a connected set, we show that the PRGM estimator under the intrinsic loss function could be Bayes with respect 
to a prior distribution in the original prior class. Theoretical results are supplemented with several examples 
and illustrations. 

Keywords: Intrinsic loss function; Bayes estimator; Robust Bayesian analysis; Posterior risk; Posterior regret 
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^ ■ 1 Introduction 

, Suppose x is a realization of a random sample X with a sampling model given by a family of densities 
{fi'W) • ^ ^ 0} with respect to a c-finite measure on a sample space x where 9 is the unknown 
parameter of interest with 6 £ Q. Let -7r(-) be a prior distribution on Q and vr(-|x) denote the posterior 
distribution of 6 given x. In standard Bayesian analysis, one needs to specify the true prior distribu- 
tion 7r(-). However, in practice, elicitation of the true prior distribution can never be done without 
error. Hence, we usually need to consider a class T of prior distributions which reflect (approximately) 
true prior behefs, i.e., the true prior distribution 7r(-) is an unknown element of T. Robust Bayesian 
analysis is designed to acknowledge such a prior uncertainty by considering the class F of plausible 
prior distributions instead of a single prior distribution vr and studying the corresponding range of 
Bayesian solutions. See Berger (1994) and Rios Insua and Ruggeri (2000) for more details. One may 
also attempt to determine an optimal estimator 5 by minimizing some measures of robustness. Several 
criteria have been proposed for the selection of procedures in robust Bayesian studies. In this paper, 
we study the maximal posterior regret method (e.g., Rios Insua and Ruggeri, 2000; Rios Insua et al., 
1995) to obtain the posterior regret gamma-minimax (PRGM) estimator of the unknown parameter 
for the one-parameter exponential family of distributions. The PRGM criterion has been used recently 
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by many people from both theoretical and practical points of view. For example, Gomez-Deniz (2009) 
investigated the use of PRGM for credibility premium estimation in Actuarial Science, Boratynska 
(2002, 2006) in insurance for collective risk model analysis, and Jafari Jozani and Parsian (2008) in 
statistical inference based on record data. 

For an observed value x, a prior distribution vr and the corresponding posterior distribution 7r(-|x), 
we denote the posterior risk of an estimate 5{x) of the unknown parameter 9 under L{9, 5) by r{x, 5) = 
E[L{9, 5{x))\x]. The Bayes estimator of 9 under the loss function L{9, 6) is then given by a d-jriX) such 
that r{x, Stt) = infs r{x, 5). 

Definition 1 The PRGM estimator of 9 under the loss function L{9,5) and a class T of prior distri- 
butions is defined as an estimator Spr such that 



where p{5tj,5) = r{x,6) — r(x,57r) is the posterior regret measuring the loss entailed in choosing the 
action 6{x) instead of the optimal Bayes action (5jr(x) (under prior n and loss L). 

In this paper, we study the construction of PRGM estimators under the so-called intrinsic loss 
functions. These loss functions shift attention from the distance between the estimator 6 and the 
true parameter value 9, to the more relevant distance between statistical models they label. More 
specifically, the intrinsic loss of using 5 as a proxy for 9 is the intrinsic distance between the true model 
f{x\9) and the model f{x\6) when 9 = 5, that is 



where d{-, •) is a suitable distance measure. In practice, intrinsic loss functions could be used as 
benchmark losses when the utility function related to the underlying statistical problem cannot be 
obtained by practitioners. A desired property of intrinsic loss functions is that they are invariant under 
one-to-one smooth reparameterizations. The invariance property of intrinsic loss functions provides 
a very convenient tool for statistical application. We show that, under suitable conditions, intrinsic 
loss functions could be used to formulate a unified set of solutions to the problem of PRGM estima- 
tion of the unknown parameter of the exponential family of distributions which is consistent under 
reparameterization, a rather obvious requirement, which unfortunately many statistical methods fail 
to satisfy. 

In Section 2, we obtain the PRGM estimator of the natural parameter 9 of the exponential family 
of distributions under the intrinsic loss function ([2]) when d(-, •) is chosen to be the Kullback-Leibler 
distance. We consider different classes of conjugate priors on the natural parameter 9 and show how to 
obtain the PRGM estimator of 9 in each class. The results are very general and provide an automated 
and unified solution to the PRGM estimation of the unknown parameter of the exponential family of 
distributions under different loss functions, including, but not limited to, quadratic, LINEX, entropy 
and Stein loss functions. 

In Bayesian statistical analysis, as pointed out by Gelman (2004), transformations of the parameter 
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typically suggest new families of prior distributions. Therefore, the usual robust Bayesian inferences are 
not invariant under reparameterizations. For example, if SpniX) is the PRGM estimator of 0, then it is 
not necessarily true that h(5pR{X)) is the PRGM estimator of r/ = h{6), when /i is a one-to-one smooth 
function. A solution to this problem is proposed in Section 3. To this end, we obtain invariant PRGM 
estimators of 9 under the intrinsic loss function and different classes of Jeffrey's Conjugate Prior (JCP) 
distributions. We show that the resulting PRGM estimates are invariant under one-to-one smooth 
transformations of 9. Theoretical results are augmented with several examples and illustrations. In 
Section 4, we provide some general results showing that, under general conditions, PRGM and intrinsic 
PRGM estimators are Bayes with respect to prior distributions in the underlying class of priors. We 
study two cases of convex classes of prior distributions as well as the case where the underlying class of 
priors depends on a hyper-parameter belonging to a connected set. We provide a sufficient condition 
under which the PRGM and intrinsic PRGM estimators are Bayes with respect to data independent 
prior distributions within the underlying class of priors. Finally, in Section 5, we give some concluding 
remarks. 

2 PRGM ESTIMATION UNDER INTRINSIC LOSS FUNCTIONS 

Suppose X is a random variable, where its distribution belongs to the one-parameter exponential family 
of distributions J-" = {f{x\9) : x(^x'~=^iG£QCl R}, with probability density function (pdf) 

/(x|0)=/3(0)t(x)e-^^(-), (3) 

where r{x) > 0, f3{9)t{x) > and 9 is the unknown real-valued natural parameter of the model. The 
density is considered with respect to the Lebesgue measure for continuous and the counting measure 
for discrete distributions. Suppose 5 is an estimate of 9 with both 9,5 £ @. We define the intrinsic loss 
function ([2]), using the Kullback-Leibler measure between f{x\9) and f{x\6), as follows 

Loss function ^ can be interpreted as the expected log-likelihood ratio in favour of the true model. 
Thus, the intrinsic loss function (j4]) not only has the desired invariance property but it is also related to 
the relevant measure of evidence in the Neyman- Pearson Lemma. Note that the intrinsic loss function 
([U is invariant under reparameterization since the parameters affect the loss function only via the 
probability distributions they label, which are independent of the particular parameterization. For 
a general reference on intrinsic losses and additional details we refer to Robert (1996) and Bernardo 
(2011). 

First, we give a lemma which identifies the intrinsic loss function for the exponential family of 
distributions. 



L{9,5) = Eg 
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Lemma 1 For the exponential family of distributions ([3]) , the intrinsic loss function Q reduces to 



^e,^) = log^^J+(<^-^)^, (5) 



where = 



Let H{t) := f3'{t)/ f3{t). A straightforward calculation shows that the posterior risk associated with 5, 
under the loss function ([5]), is 

r{x,5) = E{logp{e)\x) -logp{6{x)) + 6{x)E {H{e)\x) -E{eH{e)\x) . (6) 

The Bayes estimator of 9 can therefore be obtained by minimizing ([6]) in 5 as follows 

5^{X) = H-\E{H{e)\X)}. (7) 

Following the decreasing monotone likelihood ratio property of the densities f{x\6) in ([3]) in r{X), 
and since -E'[r(X)] = H{9), H{ ) is a decreasing function. Therefore, the Bayes estimator 5.,j{X) is 
unique. Furthermore, the posterior regret for estimating 9 using 5 instead of the optimal estimator 5^ 
is obtained by 

p{5^,5) = log ^ + {5 - 5^)H{5^). (8) 

Note that p{6tj, (5), as a function of 5-,^, decreases then increases with a unique minimum at = 6. The 
main result of this section is given in the following theorem which obtains the PRGM estimator of 9 
under the intrinsic loss function ([5]). 

Theorem 1 Let 5{x) = inf^gr (^ttI^;) and5{x) = sup^^j^ dTi{x) and suppose that S{x) and 6{x) are finite 
almost everywhere. The PRGM estimator of 9 in the exponential family ^ under the loss function 
([5]) and in the class of prior distributions T is given by 

^ _ mx)) - 6{X) H{5{X)) - log m 

Proof: First, note that 

inf sup/3((5,r5 f^) = mill < inf sup p{6t^,6), inf_ sup p{6t^,6), inf sup p{6t^,6)>. 
^ TTGr |^^<^ Trer 5<5<5 wer 5>5 Trer J 

So, we consider the following three cases: 

Case 1. When 6 < 5, we have sup^gp p{^tt, 5) = p(^, Let fi{6) = p(6, 6) = log + {5 - 6)H(6) 
with f[{5) = H{6) — H{6) < 0, following the decreasing property of H{-). Hence, fi{6) is a decreasing 
function of (5 for 5 < 5 and mis<sfiiS) = fi{S). Therefore, 

inf sup p{5tt , S) = p{S , 6) . 
'5<^ Trer 
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Case 2. For 6 > 6, we have sup^gr p{6^,S) = p{5, 5). Let f2{6) = p{6, 6) = log gfj + (5 - 6)H{6) with 
f2{S) = H{6) — H{6) > 0. Hence, f2{S) is an increasing function of 6 ior 5 > 6 and inf^>^ /2(<J) = f2{S). 
Therefore, 

inf sup p{dTt,6) = p{6,6). 
5>5 Trer 

Case 3. If 6 < 5 < 6, then sup^gp /9((57r, 5) = max{/9((^, 5), 5)}. Let /3((^) = — /2('5) where 

f^{S) = H(5) - H{5) < 0. Since f3{6) is a decreasing function of 5 with /3(^) < and /3(^) > 0, 
there exists a unique 6* G {S,6) (as the root of fsiS) = 0) such that p{6,6*) = p{5,5*). Hence, for 
6 < 6 < 6* , sup^gp p(57r, 5) = p{6,6) and for 6* < 6 < 6, sup^fz-p p{Sn , = p{^,S)- Note that, for 
6< 6 < 6, p{S,6) is a decreasing function in with inf^^^^^. sup^gp p{6t^,5) = p{6,6*) and p{5,6) is 
an increasing function in 6 with inf^,^^^^ sup^gp p{5Tr,5) = p{6,6*). Therefore, 

inf _ sup p{5Tt,5) = p{6,5*) = p{6,6*). 
5<<5<(5 7rer 

Following the above cases, we conclude that 

inf sup p((5^,(5) = inf_ sup p{5t^,5) = p{6,6*) = p{6,6*). 
(^eOTrer s<s<s vrer 

That is, the PRGM estimator of 9 is given by 6pji = 6* € {6, 6), as the solution of 

log m + 5pR{Hi6) - Hi6)) + 6Hi6) - 6H(6) = 0, 
in 5pji which results in the estimator Q. □ 



We give some applications of Theorem [TJ 

Example 1 (Normal distribution). Suppose X ~ N[p,, 1) is a normally distributed random variable 
with unknown parameter /i G M and pdf f{x\p) = -^e^ 2*-^"'^^ , — oo < x < oo. The pdf f{x\fi) belongs 

to the exponential family ^ with 6 = p, and /3{6) = e~~ . Also, H{6) = —9, and the intrinsic loss 
function ([5]) reduces to L{9,6) = ^{6 — 9)'^ which is essentially the usual squared error loss function. 
Let 5 and 5 be defined as in Theorem Ul Using ([9]), subject to the existence of 6 and S, the PRGM 
estimator of 9 in the class P of prior distributions is given by 

6pRiX) = ^(6{X) + 6{X)), 

which is also obtained in Rios Insua et al. (1995) as well as Berger (1994). 

Example 2 (Exponential distribution). Suppose X ~ Exp{a) is an exponential random variable with 
pdf f{x\a) = ^e~^^" , X > 0, where o" > is the unknown parameter. The pdf f{x\a) belongs to the 
exponential family ^ with 9 = ^, and I5{9) = 9. In this case, H{9) = 9^^, and the intrinsic loss 
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function ^ reduces to the Stein loss 



L{9, 6) = i-log^--l. 



Using ([9]), subject to the existence of 6 and 5, the PRGM estimator of 9 under the Stein loss function 
is given by 

5{X) HX) 

The PRGM estimator of a is also obtained in Example\^ 

Example 3 (Binomial distribution). Suppose X ~ Bin{n,p) is a binomial random variable with 
probability mass function (pmf) f{x\p) = — p)^~^ , where n is known, x = 0,1,..., n, and 

p G [0, 1] is the unknown parameter. The pmf f{x\p) is a member of the exponential family ([3]) with 
9 = log(i^) and f3{9) = (1 + e~^)~". We also have H{9) = which results in the intrinsic loss 
function 

i(«.*) = "{>og(^4^)+^}. (10) 
Using ([9]), subject to the existence of 5 and 6, the PRGM estimator of 9 is given by 

S(X) SjX) ] J" e^(Jf ) l+e^m \ 

dpR[X) = 1 -T— • (ll) 

In Example^ we obtain the PRGM estimator of p. 

We now consider the PRGM estimation of 9 under conjugate classes of prior distributions. For the 
exponential family ([3]) and a conjugate prior distribution 

7T^,x{9)^{mre-'\ (12) 

the posterior distribution is given by 7r{9\x) oc {I3{9)}^^°' ^ ^nd tt{9\x) = 7ra+i,x+r{x){G)- Also, 

as established by Diaconis and Ylvisaker (1979), E[H{9)\x] = -^qrp- Now, the Bayes estimator of 9 
under the intrinsic loss function ^ is obtained by (e.g., Bernardo and Smith (1994), Robert (1996) 
and Gutierrez-Pena(1992)) 

MX,^.-'(A±^). (13) 



Furthermore, the posterior regret for estimating 9 with 6{x) is p{dn, ^) = log ^(j^^+{^{x)~^n{x))^-^^^ 
Now, suppose that the prior distribution tTq^a belongs to the following class of conjugate prior distri- 
butions: 

T = {Tra,xi9) : ae[ai,a2], Ag[Ai,A2]}, 
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with suitable choices of a\ < and Ai < A2 leading to proper posterior distributions for Q. A 

^2±^ and B{b[x)) = ^i±# 

ai+l \ \ /J 0:2+1 



straightforward calculation shows that H{6{x)) = and H{5{x)) = Al+lM, Hence, we can state 



the following result. 

Lemma 2 Suppose U{t) = H~^{t) with H{t) = (3' (t)/ (3{t) . The PRGM estimate of for the exponen- 
tial family ([3]) under the intrinsic loss function ([5]) and in the class T of prior distributions is given 
by 

M±r(x)rr ( Xl+r{x) \ _ X2+r{x) jj ( \2+r{x) \ _ , ( PjUj^^)) 



02+1 \ «2+l / ai+1 \^ ai+1 

\-i+r{x) _ \2+r{x 
a2+l cti+l 



^PR^^) = ^ ^ \^f^ ^ ' ■ (14) 



Remark 1 One can also consider other classes of conjugate priors such as Ti = {7ra,\oi0) '■ ol G 
[ai,a2]j Ao is fixed} or T2 = {T^ao.xif^) ■ a = oq is fixed, X G [Ai,A2]}. The PRGM estimator of 9 in 
Ti or T2 can be obtained using (|14p and by letting Ai = A2 = Aq or a\= a2 = olq, respectively. 



Example 4 In Examplel^ letTTa^x{d) oc 0"' ^e ^'^ with the posterior distribution 'k{9\x) = TTa+i,\+x{9), 

a+l 

\+x ■ 



and 6Tt{x) = fi^- Using (I14p . the PRGM estimator of 6 under the Stein loss function L(9,6) 



I — log I — 1 m r = {iTa,x{9) : a S [01,02], A € [Ai, A2]}, with < oi < 02 and < Ai < A2 is given 
by 

In Ti, as defined in RemarkUl we have 



ri fY-. _ l«i + 1) («2 + 1) , „ / Qi + 1 \ 1 
^^^^^ V^^J 



^PRi^) = „ log 

Similarly, in T2, we have 



"0 + 1 \ , / A2 + X 



3 Intrinsic PRGM estimation 

In Section m we obtained the PRGM estimator of the natural parameter 9 of the exponential family un- 
der the intrinsic loss function. In some applications, there may be interest in finding PRGM estimation 
of the original parameter of the underlying model rather than the natural parameter 9. Unfortunately, 
like many other methods, PRGM estimators are not necessarily invariant under reparameterization. 
Although results of this nature, that are not invariant under reparameterization, can sometimes be 
interesting in theory, they tend to be less useful in practice. Indeed, it is difficult to sell to a practi- 
tioner that the PRGM estimator of h{9) is not necessarily h{5pR). In this section, we obtain PRGM 
estimators that are invariant under one-to-one smooth reparameterizations, hence the name intrinsic 
PRGM estimators. 

For the exponential family ([3]), as opposed to the well known and commonly used conjugate prior 
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(|12p . consider the following conjugate prior distribution for 9 



niM^imre-^'y^), (15) 

where lg{0) is the Fisher information for 9. Druilhet and Pommeret (2012) introduced (jl5p and referred 
to it as the Jeffrey's Conjugate Prior (JCP). It is easy to see that the JCP is invariant under smooth 
reparameterizations, and the necessary conditions on a and A in (llSp . leading to proper posterior 
distributions, do not depend on the choice of the reparameterization. The invariance property of 
JCP under any smooth and one-to-one reparameterization r] = h{9) can be shown by the following 
relationship 

Remark 2 For the exponential family since Ie{9) = —H'{9), the JCP is given by x(^) ^ 

{mre->^'y^^W(9). 

First, we give the following result. 

Lemma 3 Suppose 5^ is the Bayes estimator of the natural parameter 9 of the exponential family ([3]) 
under the intrinsic loss function Q with respect to the JCP distribution (jlSp . For every one-to-one 
smooth transformation h{9), the Bayes estimator of h{9) is h{5^). 

Proof: The proof is similar to the proof of Lemma 6.2 of Robert (1996) and hence omitted. 



Now, we state the main result of this section which can easily be proved using the invariance property 
of both the class of JCP distributions and the intrinsic loss functions under smooth reparameterization 
oi9. 

Theorem 2 Suppose 5^pf^{X) is the PRGM estimator of the unknown parameter 9 for the exponential 
family Q under the intrinsic loss function (0) with respect to a class of JCP distributions for 9. 
Then, for any one-to-one smooth transformation h{9), the PRGM estimator of h{9) is h{6Ypj^{X)). 

Proof: By definition, the PRGM estimator of h{9) in the class F'^ of JCP distributions is given by the 
solution of ^ 

inf sup p(<)t, 6) = inf sup (log ^ + {6- 6^)H{5';^)\ , 

where 5^ is the Bayes estimator of h{9). Note that p{6'^,S) = L{d^,S) where L is defined in ([5]). Now, 
using the invariance property of L and LemmaO since 6^ = h{5Tj), with J-^- being the Bayes estimator 
of ^, we have 

inf sup p{5^,5) = inf sup p{h(y5T^),5) 

= inf sup /9(/i(57r), /i(t)) 
= inf sup p{6Tr,t). 

t:h(t)=S ^^YJ 
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Therefore, if 6Yp^{X) is the PRGM estimator of 9, i.e., minimizes (in t) sup^gpj p{S-n-,t), then, the 
transform h{6^pj^{X)) is the PRGM estimator of h{6), that is, h{6Ypp) minimizes (in 5) sup^gpj p((^^, 5) 
and this completes the proof. □ 

Example 5 Suppose X ~ Exp{a) with u, x > 0. In Example\^ we showed that the intrinsic loss for 
estimating 6 = a^^ by 6 reduces to the Stein loss function 

L{9, 6) = ^--log^--l. 

Under the JCP distribution vr^>,(^) oc O'^^^e^^^, a > 1, the posterior distribution is a Gamma{a, j^) 
with Tr^{9\x) oc 0o^-^e^(^+^)^ which results in the Bayes estimator of 9 as d.,r{X) = Also, the 

intrinsic PRGM estimator of 9 under L(9, 6) is given by 



oipr[X) = 



Now, for the estimation of rj = a = ^ using 6, it is easy to see that the Bayes estimator of rj under the 
entropy loss function 

L(r/,5) = 2-log|-l, 


is given by 6Tt{X) = = s~lx)- see this, note that 7r'^(r/) oc ri^°^e^^^'^ with 7r'^(?7|2;) oc 



\ + x 



("+i)g 77 dfif]^ St^^x) = E[ri\x]. Also, the intrinsic PRGM estimator of rj is given by 



~,T-J 5{X)-6{X) _ 5{X) S(X) 



log 6{X) - log|(X) log 1(3^ - log STpr{X) ■ 

For the PRGM estimation of 9 under the Entropy loss function and its application to record data 
analysis we refer to Jafari Jozani and Parsian (2008). Similarly, if rf = —^\og9, a ^ Q, then the 
intrinsic PRGM estimator of Vj* under the LINEX loss function 

L{r]*,6*) = e'^(''*-^*) -a(ry* -5*) - 1, 

is given by 



pj 1 ( e'^('5*(^)-^W) - 1 1 1 pj 

5*^jppiX) = 5*(X) + -log{ \ = -\og5%r,{X), 

IPRV I _v )^ ^ ^\a{5*{X)-8*{X)) ] a ^^^^^ 



which is the PRGM estimator obtained in Boratynska (2006). 

For the exponential family ([3]), suppose that the prior distribution belongs to the following class of 
JCP distributions: 

r-^ = {<AW : aG[ai,a2], Ag[Ai,A2]}, (16) 

for suitable choices of ai < a2 and Ai < A2. We continue with some applications of Theorem [5] under 
the above class of priors. Similar results can be obtained in other classes of JCP distributions (see 
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Remark [T]), which we do not present here. In view of Theorem [2l and to obtain an intrinsic PRGM 
estimator, the critical condition is that the elements of the underlying class of prior distributions are 
in the form of (|15p and the underlying loss function is intrinsic. We observe that, in many cases (see 
Examples [6] and [Tj) intrinsic PRGM estimators under T"^ can be obtained using the PRGM estimators 
under the usual class F of conjugate priors with modified values of a^s and AjS in F, z = 1,2. One 
can easily check that this will happen whenever the mean- value parameter is conjugate for the natural 
parameter in the sense of Gutierrez- Pena and Smith (1995). In the one-parameter case, a sufficient 
condition for this is that the exponential family have a quadratic variance function (see Section 3.3 of 
Gutierrez-Pena and Smith (1995)). 

Example 6 In Example\^ we showed that T^^xi^) ^ 6'"~^e~^'^ and 6.„{x) = Since i^i^l^) 

is equal to tt{6\x), the posterior distribution of 9, given the usual conjugate prior TTa-i^x+xiff), the 
intrinsic PRGM estimator of 9 under the Stein loss function and the class of J CP distributions can 
be obtained using the PRGM estimator of 9 under the usual class of conjugate priors ( as in Example 
by replacing ai with ai — 1, i = 1,2. For example, the intrinsic PRGM estimator of 9 in T"^ with 
< ai < 02 and < Ai < A2 is given by 

sT-' rv\ 1 f ai Xi + X\ / f Xi + X X2+X 
SiprW = log 



a2 X2 + X J / \ 0.2 ai 

Let T{ = {T^a pi^) • ^ ^ [01102] and A = Aq}. Then, the intrinsic PRGM estimator of 9 in r{, with 
< oi < 02 and Aq > 0, is given by 

^iPRi^) = ( :: — — ) log ( — 



01 — 02/ \a2yAo + ^ 



Similarly, in F2 = {^^a si^) ^0 = 09 fixed and A G [Ai,A2]}, the intrinsic PRGM estimator of 9 



F2 , with oq > and Ai, A2 > 0, is given by 

j-ri oo , ( X2 + X 

'-^^'')=A^^°nA7TT 

Similar results can be obtained for estimating any smooth and one-to-one function of 9 under corre- 
sponding class of JCP distributions. 

Example 7 (Binomial Distribution). In Example 0, we showed that pmf of X can be written as 
f{x\9) = (")(Y^)"e-^^ with 9 = log(i^). Here Iei9) = j^^g^ and the JCP for 9 is obtained as 
^ixi^) °^ ii^Te-^^^. This results in the posterior distributiom:\9\x) cx (_^)°+"+ie-(^+^+5)^ 
Since t^'' {9\x) is equal to t:{9\x), the posterior distribution of 9, given the usual conjugate prior vr^_^]^ x+- (^)' 
the intrinsic PRGM estimator 5Yp^{X) of 9 in T'^ can be obtained using (jlip and by replacing Oj and 
Xi with Oj -|- 1 and Xi + i^, i = 1,2, respectively. Also, the intrinsic PRGM estimator of p = under 
the loss function 

f p\ ( \ — p 

L{p, (5) = p log ( I 1 (1 - p) log 
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is given by 5^pj^{X) = {1 + e'^^^C^)} i. 

4 PRGM, Intrinsic PRGM and Bayes estimators 

In this section, we provide some general results concerning the Bayesianity of the PRGM and intrinsic 
PRGM estimators of 6 for the exponential family distribution ([3]) under the intrinsic loss function ([5]) 
with respect to priors in the underlying class of prior distributions. The results are only presented for 
PRGM estimators of 6, but they can also be used for intrinsic PRGM estimators by simple modifica- 
tions. Our framework in this section closely resembles the one introduced by Rios Insua et al. (1995), 
who considered similar problem for the quadratic loss function. Results of this nature are also obtained 
by Zen and DasGupta (1993) under the quadratic loss function for the binomial distribution. Several 
of the following preliminary results and detailed proofs are reported here for the sake of completeness. 
The idea is to check the continuity of the underlying Bayes estimator with respect to the prior. Similar 
to Rios Insua et al. (1995) we study two cases, when (a) the class of prior distributions is convex, or 
(b) the underlying class of prior distributions depends on a hyper-parameter belonging to a connected 
set. 

First, consider the situation where the class F of priors is convex. That is, if 7ro,7ri € F, then 
TTt = t-KQ -|- (1 — t)7ri belongs to F, for any t S [0, 1]. Suppose that X is a random variable whose density 
belongs to the family of distributions Let ip{t) = H[5t^X^)) which is a decreasing function of 5-,^^ 
for any t £ [0, 1]. In the next lemma we show that ^(t) is a continuous function in its domain t € [0, 1]. 

Lemma 4 Suppose ipit), the posterior expectation of H{9) = ^j^y when nt = tvro + (1 — t)7ri, t £ [0, 1], 
is finite. Then, tp[t) is continuous in t, t £ [0, 1]. 

Proof. Let Oj = Jq H{6)-Ki{6)f{x\6)d6, mi{x) = fQ-Ki(9)f{x\9)d9 for i = 1,2, and suppose that a^s 
and mjS exist and are finite. Then 

_ t /e H{9) f{x\9) 7ro{9)d9 + (1 - t) H{9) f{x\9) 7r^{9)d9 

t /e f{x\e)7To{0)d9 + (1 - t) /e f{x\9)7Tii9)d9 
_ taQ + (1 - t)ai 
tmo{x) + (1 — t)mi{x) ' 

which is a continuous function of t, t £ [0, 1]. □ 

Now, we use the continuity of iplt) to prove that, under the conditions of Lemma IU the PRGM 
estimator dpR is Bayes if the class of priors is convex. 

Theorem 3 Suppose T is a convex class of prior distributions on the unknown parameter 9 of the 
exponential family of distributions ([3]). Then, there exists a prior distribution tt £ V such thatdpR = 5-,^, 
where 5pr is defined in ([9]). 

Proof. Following the definition of 5_ and 5, consider a small enough e > and two prior distributions 
TTo, TTl € F such that 

^■KQ <5_ + e < 5 PR < ^ - e < Jjri • 
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Since H(-) is a decreasing function, then 



Hi6^,) < H{6 - e) < H{5pr) < H{6 + e) < H{6^,). 

Let TTt = tvro + (1 - t)vri, t G [0,1] and define V(0 = H{6nJ. Note tliat, V(0) = H{6^J and 
V'(l) = H{5t^^). Now, from Lemma HI the continuity of "ip{t) in t shows that there exists a t* G [0, 1] 
such that ip{t*) = H{5t^^,) = H{5pji), which completes the proof. □ 

A shortcoming of the result in Theorem [3] is that it is not applicable to the cases where the class 
of prior distributions depends on a hyper-parameter whose range is connected. For this case, we prove 
Lemma and Theorem U] which are simple extensions of Lemma 3.2 and Proposition 3.2 of Rios Insua 
et al. (1995). The proof of Lemma [5] is essentially similar to the proof of Lemma 3.2 of Rios Insua et 
al. (1995). The same is true of TheoremU) Nonetheless, we provide the proofs in the Appendix for the 
sake of completeness. Let 

^ f^H{e)fix\9)7r{9)d9 ^ r(vr) 

Consider d{-K,'ir') = supQ |vr(0) — Tr'{9)\ to be the usual /qo distance between prior densities vr and n' , 
where H{t) = fi'{t)/ j3{t) is defined as before. 

Lemma 5 Suppose that Jq \H{9)\f{x\6)d6 exists and it is finite. Then, tpi''^) continuous in vr, in 
the topology generated by the /qo distance. 

Theorem 4 Let T = {vr^ : a G A}, where A is a connected set and vr^ 's are densities. Under the 
conditions of Lemma and the assumption that Un ^ ot implies d{TTa„,TTa) 0, there exists a prior 
distribution tt G F such that 6pr = 5-,^, that is, the PRGM estimator @ is Bayes. 

In the following lemma, we provide a sufficient condition under which the PRGM (or intrinsic 
PRGM) estimator is Bayes with respect to the same prior in the underlying class of prior distribution, 
regardless of the observed value of x. 

Lemma 6 Let F = {tTq : a G [ai,a2]} be the class of prior distributions. Suppose the Bayes estimator 
^'(a,x) = H^^ {E[H {9)\x)} is a differentiable function of the hyper-parameter a and the observed value 
X. Assume that we are under the conditions of Theorem^ If 



dx ' dx 



^(ai, x) g(^(ai, x)) - ^{a2,x) H{^{a2,x)) - log gg|g"^gj 
Hi^if{ai,x))-H{^{a2,x)) 



(18) 



has a constant solution in a, then there is a data independent prior tTq, G F resulting in the PRGM 
estimate as the Bayes estimate of the natural parameter 9 of the exponential family ([3]) under the 
intrinsic loss function 

Proof: Under the conditions of Theorem HJ there exists a solution a{x) such that the PRGM estimator 
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([9]) is Bayes with respect to the prior TTa^x) ^ T under the intrinsic loss function That is, 

^{a^,x) g(^(ai, x)) - ^/(a^, x) H{^{a,,x)) - log 

Now, differentiating the equation with respect to x leads to 
d doiix) d 

_ d [ ^(ai,x)g(M^(ai,x))-^(a2,x)g(^(a2,x))-log gg;:^^gj 

If a{x) is data independent, i.e., a{x) = a, then ^"'^^^ = 0. Now, the desired value for a is the constant 
solution to the equation (jlSp leading to a data independent prior for the PRGM estimator to be Bayes. 
□ 

Example 8 In ExampleUl the condition (jlSp reduces to the condition (5) in Proposition 3.3 of Rio s 
Insua et al. (1995) as follows 

Now, consider the class T = {i^aM '■ o G [ai)a2], Aq is fixed } of conjugate priors where tTq^Ao giwen 
6?/ (|12p 6 = and f3{9) = e~^^/^. //ere, t/ie Bayes estimator of 6 is given by ^{a,X) = 6t^^ xi-^) ~ 
■ /t is easy to see that, the PRGM estimator of 6 given by 



2\ai + l a2 + lj' 

is Bayes with respect to the data independent prior TTa*,Xo ^ ^ where a* is given as the solution to the 
following equation 

+ 



a* + 1 ai + 1 02 + 1 
That is, a* = G [ai,a2] and 6pr{X) = |^ = <5,,.,,(^). 

Example 9 In Example^ the condition (fTSj) reduces to 



5a; ' 9x 1 



1 



Now, consider the class Ti = {TTa,\o{G) ■ a € [Q!i,a2], Aq is fixed} of conjugate priors on 9. Here, the 
Bayes estimator of 9 with respect to the prior iTa^X(-){9) is '!'(«, X) = S-^-^ ^^^{X) = . The PRGM 
estimator of 9 is then Bayes with respect to a data independent prior i^a*,\o^^) G Ti, if there exists a 
data independent solution a* to the equation 

a* + 1 , /ai + 1 \ (ai + 1) (a2 + 1) 1 

- log ' 



{Xo + Xf ^\a2 + l) ax-a2 (Aq + X) 
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A straightforward calculation shows that 

^ (ai + 1) (a2 + 1) , /ai + l\ 

a = log — - - 1 € [ai, a2j. 

ai - 02 V^z + 1/ 

Therefore, the PRGM estimator of 6 under the Stein loss function can he obtained as the Bayes esti- 
mator of 9 with respect to the prior distribution T^a*,\Q^^) ^ -'^i follows 

Similarly, in Example\^ one can easily show that the intrinsic PRGM estimator 5^p^{X) is the Bayes 
estimator of 6 under the Stein loss function with respect to the prior distribution vr^** ^^^^ G Tf, when 
^** ~ a\-a2 ^^^^ ^^^^ 1/a** is the logarithmic mean ofl/ai and l/a2, and a** G [ai,a2]- 



5 Concluding Remarks 

Invariant estimators are usually demanding in practice. In this paper, we have provided general results 
concerning the PRGM estimation of the natural parameter of the one-parameter exponential family of 
distributions under intrinsic loss functions. The PRGM estimators are shown to be invariant to one- 
to-one smooth reparameterizations under intrinsic loss functions and the class of Jeffrey's conjugate 
prior distributions. Moreover, when the class of priors are convex or dependant on a hyper-parameter 
belonging to a connected set, we show that the obtained PRGM estimators could be Bayes with respect 
to prior distributions in the underlying class of priors. Several examples are provided to clarify the 
results. 
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6 Appendix 

6.1 Proof of Lemma [3] 

Suppose that d{-K,iT') < e. Then, for all S G, Tr{9) — e < it' {9) < i:{9) + e, and so 

f{x\9)T:{9) - f{x\9)e < fix\9)7T'{9) < f{x\9)7T{9) + f{x\9)e. (19) 
Upon integrating (|19p over 9 we get 

s(7r)-e / fix\9)d9 <siTr') <siTr) + e [ f{x\9)d9. 
Je Je 
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Let ^0 e G be such that H{e) = > for ah < ^o, and H{e) < for ah 6 > Oq. For 6 < Oq, 
multiplying (jl9p by H{6) > results in 

H{6)fix\e)7rie) -eH{e)f{x\e) < H{e)f{x\ey{e) < H{e)f{x\e)^{e) + eH{e)f{x\e), (20) 

while for 9 > 9q we have 

H{6)f{x\6)7ri9) + eH{9)f{x\9) < H {9) f {x\ey {9) < H{9)f{x\9)7r{9) - e H {9) f {x\9) . (21) 
Using (pOj) and ([2T]) and integrating over 9, leads to 

r{9)-e[ \H{9)\f{x\9)d9 <r'{9) <r{9) + e [ \H{9)\fix\9)d9. 
Je Je 

Since we assumed that Jq \H{9)\f{x\9)d9 = Ki < oo, then Jq f{x\9)d9 = K2 < 00, and 

r(7r) — eKi ^ r(7r') ^ r(7r) — eKi 
s{tt) + ~ s(vr') ~ s(7r) — 

If e ^ 0, then ig) ^ . Therefore, V(vr') ^ V'(^)- 
6.2 Proof of Theorem H 

We consider ttq = vToy and vri = tTq^. Due to the connectedness of A, there is a continuous path 
g{t) G A,t G [0,1] such that g{0) = ao, and g{l) = ai. Let V'(t) = V'K(t)) = H{6^^^^^),t G [0,1]. 
is a continuous function in t, so there is t* G [0, 1] such that ip(t*) = ^(vrp/j) leading to 'n'g^f) G F as 
the prior distribution we were looking for. 
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